Validation and reliability testing of a rating scale for objective assessment of performance in laparoscopic appendicectomy surgery.

BACKGROUNDS: To achieve a competency-based training paradigm, the ability to obtain reliable and valid quantitative assessments of intraoperative performance is required. Through this, weaknesses can be identified and practiced, and competency assessed. This study aimed to determine the validity and reliability an objective evaluation tool for assessment of performance in laparoscopic appendicectomy (LA). METHODS: A prospective single-blinded observational study design was used. Videos of inexperienced (performed <10 LAs) and experienced (performed >100 LAs) surgeons performing LA surgery were collected. Surgical performance during each recording was rated by two independent, blinded expert surgeons using the LA Rating Scale (LARS) and the modified Objective Structured Assessment of Technical Skill (OSATS) scale. RESULTS: The intraclass correlation coefficient (ICC) for LARS was 0.95 (95%CI 0.83-0.98). The ICC for each step ranged from 0.48 to 0.90, and the test-retest ICC for LARS was 0.91 (95%CI 0.69-0.98). Significant differences (P < 0.001) between median performance scores as rated by LARS were observed between the inexperienced and experienced surgeons. A Spearman's correlation coefficient of 0.87 (P < 0.001) was observed between LARS performance scores and modified OSATS scores. CONCLUSION: LARS demonstrated excellent inter-rater and test-retest reliability, and construct and concurrent validity and can be used to quantitatively evaluate performance during LA. This can potentially allow specific weaknesses to be identified and improved upon through deliberate practice. Progress can be tracked through re-evaluation and scores of expert surgeons can be used as performance goals for credentialing in LA.

Development of a rating scale for objective assessment of performance in laparoscopic appendicectomy surgery.

BACKGROUNDS: Laparoscopic appendicectomy (LA) is the most common index procedure for junior surgical trainees. Despite the shift towards competency-based training, there is no method of quantitatively assessing performance during LA. This study aimed to obtain expert consensus regarding the items required to create a LA Rating Scale (LARS). METHODS: A list of steps required for LA surgery, as well as descriptors of "poor", "average" and "excellent" performance for each of these steps were created for potential inclusion into an objective assessment tool for LA surgery. Using a Delphi method, 20 experts from multiple institutions rated on a Likert scale from 1 to 5 the suitability of these steps and descriptors of performance. Responses were obtained until consensus (Cronbach's α > 0.8) was achieved. RESULTS: Fifteen experts participated in the study. Consensus was achieved for all items during the first iteration of the Delphi with a Cronbach's α of 0.96. The Cronbach's α for the steps was 0.87 and 0.92 for the descriptors of performance. Steps and descriptors of performance that >80% of experts rated as ≥4 were used to create the final LARS tool. CONCLUSION: Multi-institutional expert consensus was obtained regarding the steps and, for the first time, descriptors of performance for LA, demonstrating their face and content validity, as well as generalisability. Subsequently, the LARS tool was created that can be used to quantitatively assess intra-operative performance. This instrument can be used to identify weaknesses in performance and facilitate deliberate practice, thus shifting training in LA to a competency-based approach.

Implementation of a Surgical Simulation Care Pathway Approach to Training in Emergency Abdominal Surgery.

BACKGROUND: Simulation-based care pathway approach (CPA) training is a novel approach in surgical education. The objective of the present study was to determine whether CPA was feasible for training surgical residents and could improve efficiency in patients' management. A common disease was chosen: acute appendicitis. METHODS: All five junior residents of our department were trained in CPA: preoperative CPA consisted in virtual patients (VPs) presenting with acute right iliac fossa pain; intraoperative CPA involved a virtual competency-based curriculum for laparoscopic appendectomy (LAPP); finally, post-operative VP were reviewed after LAPP. Thirty-eight patients undergoing appendectomy were prospectively included before (n = 21) and after (n = 17) the training. All demographic and perioperative data were prospectively collected from their medical records, and time taken from admission to management was measured. RESULTS: All residents had performed less than 10 LAPP as primary operator. Pre- and intraoperative data were comparable between pretraining and post-training patients. Times to liquid and solid diet were significantly reduced after training [7 h (2-20) vs. 4 (4-6); P = 0.004, and 17 h (4-48) vs. 6 (4-24); P = 0.005] without changing post-operative morbidity [4 (19%) vs. 0 (0); P = 0.11] and length of stay [48 h (30-264) vs. 44 (21-145); P = 0.22]. CONCLUSIONS: CPA training is feasible in abdominal surgery. In the current study, it improved patients' management in terms of earlier oral intake.